On staleness and the delivery of web pages

A popular technique to improve the scalability of a web based system is caching at proxy servers. Caching has the drawback that a cached page becomes stale when the page is updated at the web server. In some cases, staleness may not be completely avoided because the server may not wish to expend the processing and communication resources required to transmit all the updates immediately. In general, if updates are transmitted less frequently, the staleness will tend to increase, but the amount of resources consumed will be reduced. The tradeoff between resource consumption and staleness is investigated. A measure of staleness is defined and optimization problems are formulated. The solutions to these problems allow one to come up with an optimal strategy for transmitting page updates. Numerical examples showing the resource consumption/staleness tradeoff are presented.